Skip to content

test(e2e): enable pipelining on p2p tests#23070

Merged
PhilWindle merged 4 commits into
merge-train/spartanfrom
spl/e2e-p2p-pipelining
May 11, 2026
Merged

test(e2e): enable pipelining on p2p tests#23070
PhilWindle merged 4 commits into
merge-train/spartanfrom
spl/e2e-p2p-pipelining

Conversation

@spalladino

Copy link
Copy Markdown
Contributor

Toggle pipelining on all e2e p2p tests

@spalladino spalladino added the ci-no-fail-fast Sets NO_FAIL_FAST in the CI so the run is not aborted on the first failure label May 7, 2026
@spalladino spalladino force-pushed the spl/e2e-p2p-pipelining branch from dddd3c6 to 4fa67ac Compare May 8, 2026 14:18
@PhilWindle PhilWindle merged commit 61a23c9 into merge-train/spartan May 11, 2026
14 checks passed
@PhilWindle PhilWindle deleted the spl/e2e-p2p-pipelining branch May 11, 2026 16:42
spalladino pushed a commit that referenced this pull request May 15, 2026
… pipelining (#23302)

## Summary

Fixes the `e2e_p2p_broadcasted_invalid_block_proposal_slash` failure
that has been blocking the `merge-train/spartan` train (run
https://github.com/AztecProtocol/aztec-packages/actions/runs/25896899879,
test log http://ci.aztec-labs.com/2bf4e2cd2d9e7944).

The test creates the malicious proposer first (auto-starting its
sequencer) and only later creates the honest nodes and waits for P2P
mesh. Under `enableProposerPipelining: true` (turned on for this test by
#23070), the malicious proposer is selected for the very next slot,
builds + broadcasts the invalid proposal one slot ahead, and lands the
broadcast before the honest validators have joined the mesh. They then
reject it at the gossipsub `checkpoint_proposal_validator` with
`Penalizing peer for invalid slot number` (since their target slot has
already moved past), so the `state_mismatch` slashing path never runs.
The malicious sequencer then gets stuck on the failed publish (`Awaiting
pending L1 payload submission`) and never proposes again before the test
times out on `awaitOffenseDetected`.

This is the same race that #23070 fixed in
`duplicate_proposal_slash.test.ts`; the same pattern is applied here:

- Create both the invalid proposer and the honest nodes with
`dontStartSequencer: true`.
- After P2P mesh connectivity + committee formation, use
`advanceToEpochBeforeProposer` to land one epoch before an epoch where
the invalid proposer is scheduled.
- Start all sequencers, then `advanceToEpoch(targetEpoch, { offset:
-AZTEC_SLOT_DURATION })` so the malicious slot fires while every node is
online and at the same wall-clock slot.
- After `awaitOffenseDetected` on one node, poll `getSlashOffenses`
across **all** nodes for `BROADCASTED_INVALID_BLOCK_PROPOSAL` — under
pipelining a given receiver may have already advanced past the build
slot when the proposal arrives, so we need to catch whichever node was
still in the build slot.

The on-chain slash assertion (`rollup.listenToSlash`) is preserved
unchanged.

Full failure analysis:
https://gist.github.com/AztecBot/39b69c1117f419145938ccd2c198f8e9

## Test plan

- CI: `e2e_p2p_broadcasted_invalid_block_proposal_slash` passes on
`merge-train/spartan`.
- Local `./bootstrap.sh ci` / `fast` / `build` are not runnable in this
container (no Docker socket and `$HOME` not writable for the container
UID — `yarn install` fails on `corepack` mkdir, parallel-bootstrap can't
create `~/.parallel`). Fix is a direct port of a pattern already
shipping green on `next` via the sibling
`duplicate_proposal_slash.test.ts`.


ClaudeBox log: https://claudebox.work/s/06a4929a1971beaf?run=1
rangozd pushed a commit to rangozd/aztec-packages that referenced this pull request May 16, 2026
BEGIN_COMMIT_OVERRIDE
fix(test): warp L1 forward when proposer scan hits EpochNotStable
(AztecProtocol#22967)
test(e2e): fail epochs tests on proposer-rollup-check-failed (AztecProtocol#22965)
fix: grafana switch to aztec_status="proposed" (AztecProtocol#22978)
chore: update benchmark scraper (AztecProtocol#22984)
test(e2e): migrate simple epoch tests to pipelining (AztecProtocol#22973)
chore: remove top-level yarn.lock (AztecProtocol#22987)
refactor(archiver)!: unify L2BlockSource checkpoint lookups via query
objects (AztecProtocol#22933)
fix(sequencer): bounded sweep instead of event scan for governance
proposal check (AztecProtocol#22989)
fix(docs): allow webapp-tutorial yarn install to populate empty lockfile
in CI (AztecProtocol#23000)
test(e2e): enable pipelining in l1-reorgs and mbps redistribution tests
(AztecProtocol#23009)
fix(archiver): restore pending block height metric under pipelining
(AztecProtocol#22994)
chore(p2p): remove skipped validation result option (AztecProtocol#23034)
refactor(p2p)!: remove slow tx collection flow (AztecProtocol#22878)
chore(spartan): add next-net-clone environment config (AztecProtocol#22995)
chore(sequencer): add context to proposer-rollup-check-failed logs
(AztecProtocol#23071)
test(e2e): wait for archiver sync before asserting pipelining (AztecProtocol#22997)
refactor(node-rpc)!: remove deprecated AztecNode methods and
L2BlockSource tip helpers (AztecProtocol#22934)
feat(p2p): detect and track announce IP changes at runtime (AztecProtocol#22405)
test: mark tx_stats_bench 10 TPS as flake-retryable on
merge-train/spartan (AztecProtocol#23083)
fix(sequencer): bind vote-only multicalls to target slot under
pipelining (AztecProtocol#23090)
feat(sequencer): build optimistically across pruning epoch boundary
(AztecProtocol#23056)
fix(sequencer): use chainTipsOverride.pending for log context (AztecProtocol#23098)
test(e2e): relax post-boundary slot assertion in
epochs_proof_at_boundary (AztecProtocol#23108)
fix(bb-prover): pool long-lived bb verifier processes instead of
spawning per-call (AztecProtocol#23093)
fix(sequencer): anchor fee asset price modifier to predicted parent
(AztecProtocol#23113)
chore: error log when L1 head timestamp drifts (AztecProtocol#22947)
fix(sequencer): override full parent checkpoint cell in pipelined
simulation (AztecProtocol#23073)
test(e2e): enable pipelining on missed l1 slot test (AztecProtocol#23068)
fix: more robust metrics reporting in IRM monitor (AztecProtocol#23038)
fix: preserve LMDB slashing protection (AztecProtocol#23145)
test(e2e): enable pipelining on p2p tests (AztecProtocol#23070)
fix(archiver): move L2 tips cache refresh out of write transactions
(AztecProtocol#23110)
test(e2e): fix data_withholding_slash flake by freezing L1 across
restart (AztecProtocol#23162)
fix(validator): include proposed checkpoint out-hashes when validating
checkpoint proposals (AztecProtocol#23119)
refactor(config): drop nested config option, flatten l1Contracts
(AztecProtocol#23143)
test(e2e): bump bash TIMEOUT for e2e_p2p/add_rollup to match jest 20m
(AztecProtocol#23177)
fix(p2p): chunk archive of mined txs on block finalization (A-969)
(AztecProtocol#23085)
fix(p2p): stream tx pool hydration to bound startup memory (A-968)
(AztecProtocol#23086)
chore: remove orphan --archiver flag usages from start invocations
(AztecProtocol#23186)
feat(ci): daily merge-train/spartan stale-PR notifier (AztecProtocol#23189)
fix: preserve contract artifact permissions (AztecProtocol#23174)
fix(ci3): accept slashes in /list/<path:key> for merge-train
history (AztecProtocol#23160)
feat(ci): route merge-train/spartan flake notifications to
#team-alpha-ci (AztecProtocol#23219)
fix(cheat-codes): wait for post-warp L2 block in warpL2TimeAtLeastTo
(AztecProtocol#23213)
feat: slash attesters signing over bad checkpoints (AztecProtocol#23180)
refactor(prover-client): split orchestrator into sub-tree + top-tree
pair (AztecProtocol#22996)
fix(srs): retry transient CRS HTTP downloads with exponential backoff
(AztecProtocol#23244)
refactor(p2p): remove old reqresp mode (AztecProtocol#23158)
docs(sequencer-client): rewrite top-level and timing READMEs (AztecProtocol#23149)
fix(aztec-node): include upcoming checkpoint's L1 to L2 messages in
simulatePublicCalls (AztecProtocol#23163)
END_COMMIT_OVERRIDE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-no-fail-fast Sets NO_FAIL_FAST in the CI so the run is not aborted on the first failure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants